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Electromagnetic  Induction  and  Magnetic  Sensor  Fusion  for 
Enhanced  UXO  Target  Classification 

Naval  Research  Laboratory 

29  June  2000 


1.  Introduction 

1.1  Background  Information 

Buried  unexploded  ordnance,  UXO,  is  one  of  the  Department  of  Defense’s  most  pressing 
environmental  problems.  Not  limited  to  active  ranges  and  bases,  UXO  contamination  is  also 
present  at  DOD  sites  that  are  dormant  and  in  areas  adjacent  to  military  ranges  that  are  under  the 
control  of  other  government  agencies  and  the  private  sector. 

Traditional  methods  for  buried  UXO  detection,  characterization  and  remediation  are  labor- 
intensive,  slow  and  inefficient.  Typical  detection  and  characterization  methods  rely  on  hand¬ 
held  detectors  operated  by  explosive  ordnance  disposal  (EOD)  technicians  who  slowly  walk 
across  the  survey  area.  This  process  has  been  documented  as  inefficient  and  marginally 
effective.^  In  addition,  a  large  portion,  approaching  70%  in  some  cases,  of  the  total  budget  of  a 
typical  remediation  effort  is  spent  on  digging  targets  that  do  not  turn  out  to  be  UXO. 

The  Environmental  Security  Technology  Certification  Program,  ESTCP,  has  supported  the  Naval 
Research  Laboratory  in  the  development  of  the  Multi-sensor  Towed  Array  Detection  System, 
MTADS,  to  address  these  deficiencies.  The  MTADS  incorporates  both  cesium  vapor  full-field 
m^netometers  and  pulsed-induction  sensors  in  linear  arrays  that  are  towed  over  survey  sites  by 
an  all-terrain  vehicle.  Sensor  positioning  is  provided  by  state-of-the-art  Real  Time  Kinematic 
(RTK)  GPS  receivers.  The  survey  data  acquired  by  MTADS  is  analyzed  by  an  NRL-developed 
Data  Analysis  System  (DAS).  The  DAS  was  designed  to  locate,  identify  and  categorize  all 
military  ordnance  at  its  maximum  self-burial  depth.  It  is  efficient  and  simple  to  operate  by 
relatively  untrained  personnel. 

The  performance  of  the  MTADS  has  been  demonstrated  at  a  number  of  prepared  sites  and  live 
ranges  over  the  past  two  years.^'*'  It  can  detect  and  locate  ordnance  with  accuracies  on  the  order 
of  15  cm.^  However,  even  with  careful  mission  planning  and  preliminary  training  there  are  still 
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significant  numbers  of  non-ordnance  targets  selected.  Thus,  more  effective  discrimination 
algorithms  are  required.  This  program  was  designed  to  exploit  the  dependence  on  target  shape  of 
the  pulsed-induction  response  to  enhance  the  discrimination  capability  of  MTADS. 

1.2  Official  DOD  Requirement  Statement 

The  Navy  Tri-Service  Environmental  Quality  Research  Development  Test  and  Evaluation 
Strategic  Plan^pecifically  addresses  under  Thrust  Requirements  1.A.1  and  1.A.2,  the 
requirements  for  improved  detection,  location  and  removal  of  UXO  on  land  and  rmder  water. 

The  index  numbers  associated  with  these  requirements  are  l.I.4.e  and  l.III.2.f.  The  priority  1 
rankings  of  these  requirements  indicate  that  they  address  existing  statutory  requirements, 
executive  orders  or  significant  health  and  safety  issues.  Specifically  the  requirements  document 
states: 


There  are  more  than  twenty  million  acres  of  bombing  and  target  ranges  under  DOD 
control  Of particular  concern  for  the  Navy  are  the  many  underwater  sites  which  have 
yet  to  be  characterized.  Each  year  a  significant  fraction  (200,000-500,000  acres)  of 
these  spaces  is  returned  to  civilian  (Private  or  Commercial)  use.  All  these  areas  must  be 
surveyed  for  buried  ordnance  and  other  hazardous  materials,  rendered  certified  and  safe 
for  the  intended  end  use.  This  is  an  extremely  labor  intensive  and  expensive  process, 
with  costs  often  far  exceeding  the  value  of  the  land....  Improved  technologies  for  locating, 
identifying  and  marking  ordnance  items  must  be  developed  to  address  all  types  of  terrain, 
such  as  open  fields,  wooded  areas,  rugged  inaccessible  areas,  and  underwater  sites 

The  MTADS  addresses  all  aspects  of  the  Tri-Service  Requirements  for  land-based  buried  UXO. 

It  is  designed  to  survey  large  sites  rapidly  and  efficiently,  with  commensurate  economic  benefits. 
Moreover,  it  is  capable  of  detecting  all  classes  of  buried  UXO  at  their  likely  self-burial  depths. 
The  system  will  correctly  locate  buried  targets,  determine  their  burial  depths,  classify  the  likely 
ordnance  size,  provide  for  future  target  way  pointing,  as  well  as  create  GIS -compatible  target 
output  maps  and  sorted  target  tables. 

1.3  Objective  of  the  Demonstration 

This  was  the  final  demonstration  of  the  modified  MTADS  incorporating  the  analysis  techniques 
described  below.  We  showed  in  the  first  demonstration  of  this  program,  the  Requirements 
Demonstration  conducted  on  the  NRL  Test  Field  at  Blossom  Point,  that  these  methods  are  useful 
tools  for  target  classification.  This  Final  Demonstration  was  designed  to  use  these  tools  on  a 
real-world  site  and  quantify  the  benefits  obtained  from  their  use. 

1.4  Regulatory  Issues 

There  were  no  external  regulatory  issues  impacting  this  demonstration.  All  activities  described 
here  occurred  On  a  government  test  range  and  were  governed  by  a  locally  approved  Health  and 
Safety  Work  Plan. 

1.5  Previous  Testing  of  the  Technology 

An  initial  test  of  the  technology  described  in  this  report,  designated  the  Requirements 
Demonstration,  was  conducted  in  February  1999  at  the  NRL  Test  Site'^  at  the  Blossom  Point 
Facility.  Results  of  that  test  will  be  described  in  following  sections  of  this  report. 
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2.  Technology  Description 


2.1  Background  and  Applications 

The  standard  MTADS  technology  has  been  described  in  detail  previously.  Briefly,  the  system 
hardware  consists  of  a  low-magnetic-signature  vehicle  that  is  used  to  tow  linear  arrays  of 
magnetometer  and  pulsed-induction  sensors  to  conduct  surveys  of  large  areas  to  detect  buried 
UXO.  The  MTADS  tow  vehicle,  manufactured  by  Chenowth  Racing  Vehicles,  is  a  custom-built 
off-road  vehicle,  specifically  modified  to  have  an  extremely  low  magnetic  signature.  Most 
ferrous  components  have  been  removed  from  the  body,  drive  train  and  engine  and  replaced  with 
non-ferrous  alloys. 

The  MTADS  magnetometers  are  Cs-vapor  fiill-field  magnetometers  (Geometries  Model 
822ROV).  An  array  of  eight  sensors  is  deployed  as  a  magnetometer  array.  The  time-variation  of 
the  Earth’s  field  is  measured  by  a  ninth  sensor  deployed  at  a  static  site  removed  from  the  survey 
area.  These  data  are  used  to  correct  the  survey  magnetic  readings.  The  pulsed-induction  sensors 
(specially  modified  Geonics  EM-6  Is)  are  deployed  as  an  overlapping  array  of  three  sensors.  The 
sensors  employed  by  MTADS  have  been  modified  to  make  them  more  compatible  with  vehicular 
speeds  and  to  increase  their  sensitivity  to  small  objects. 

The  sensor  positions  are  measured  in  real-time  (5  Hz)  using  the  latest  RTK  GPS  technology.  All 
navigation  and  sensor  data  are  time-stamped  and  recorded  by  the  data  acquisition  computer  in 
the  tow  vehicle.  The  DAS  contains  routines  to  convert  these  sensor  and  position  data  streams 
into  anomaly  maps  for  analysis. 

The  standard  MTADS  analysis  method  has  also  been  described  previously.  The  magnetometry 
data  has  been  very  successfully  modeled  using  a  dipole  response.  We  routinely  recover  target 
x,y  positions  to  within  15  cm  and  target  depths  to  ±  20%.^  Within  the  signal  to  noise  ratio  of  the 
MTADS,  we  see  no  residual  signature  attributable  to  higher  moments.*'*  The  pulsed-induction 
modeling  has  been  less  successful.  The  standard  algorithm  is  based  on  a  sphere  model  and  does 
not  well  represent  the  signatures  we  obtain.  We  have  discussed  the  deficiencies  of  this  model 
and  proposed  an  ordnance  model  based  on  a  prolate  spheroid. 

This  program  was  organized  around  the  premise  that  classification  based  on  shape  is  central  to 
the  problem  of  discriminating  between  unexploded  ordnance  (UXO)  and  clutter.  Most  UXO  fit  a 
specific  profile:  they  are  long  and  slender  with  typical  length-to-diameter  aspect  ratios  of  four  or 
five.  Many  clutter  items,  on  the  other  hand,  do  not  fit  this  profile.  Using  pulsed-induction 
sensor  data,  we  have  developed  a  model-based  estimation  procedure  to  determine  whether  or  not 
a  target  is  likely  to  be  a  UXO  item.  The  model  relies  on  exploiting  the  dependence  of  the 
induced  field  on  target  size,  shape  and  orientation. 

The  EM61  is  a  time  domain  instrument.  It  operates  by  transmitting  a  magnetic  pulse  that  induces 
currents  in  any  nearby  conducting  objects.  These  currents  produce  secondary  magnetic  fields  that 
are  measured  by  the  sensor  after  the  transmitter  pulse  has  ended.  The  sensor  response  is  the 
voltage  induced  in  the  receiver  coil  by  these  secondary  fields,  and  is  proportional  to  the  time  rate 
of  change  of  the  magnetic  flux  throu^  the  coil.  The  sensor  integrates  this  induced  voltage  over  a 
fixed  time  gate  and  averages  over  a  number  of  pulses.  An  illustration  of  the  magnitude  and 
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direction  of  the  field  transmitted  by  the  MTADS  array  is  shown  in  Figure  1 .  Note  that  the  field 
experienced  by  an  object  directly  below  the  array  is  substantially  different  than  an  object  in  fi-ont 
or  behind  the  array.  The  implications  of  this  for  target  signatures  will  be  seen  in  Section  5  of  this 
report. 


3  Overlapped  1-m  Coils 


The  model  to  be  used  in  this  demonstration  has  been  jointly  developed  by  NRL  and  AETC,  Inc. 
and  has  been  described  recently.*'*’’^  Briefly,  it  relies  upon  the  fact  that  the  EM61  signal  is  a 
linear  function  of  the  flux  through  the  receiving  coil.  The  flux  is  assumed  to  originate  from  an 
induced  dipole  moment  at  the  target  location  given  by: 


m  =  UBU^  Ho 


(1) 


where  H#  is  the  peak  primary  field  at  the  target,  U  is  the  transformation  matrix  between  the 
coordinate  directions  and  the  principal  axes  of  the  target,  and  5  is  an  empirically  determined, 
effective  magnetic  polarizability  matrix.  For  any  arbitrary  compact  object,  this  matrix  can  be 
diagonalized  about  three  primary  body  axes  and  written  as: 


0  0 
Py  0 
0 
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(2) 


The  relative  magnitudes  of  these  p’s  are  determined  by  the  size,  shape,  and  composition  of  the 
object  as  well  as  by  the  transmit  pulse  and  fixed  time  gate  of  the  EM61.  Different  time  gates 
may  result  in  different  values  and  different  relative  values  of  these  P’s  for  a  given  object.  The 
transformation  matrix  contains  the  angular  information  about  the  orientation  of  these  body  axes. 

For  an  axisymmetric  object,  5  is  a  diagonal  tensor  with  only  two  unique  coefficients, 
corresponding  to  the  longitudinal  (fii)  and  transverse  (fit)  directions: 


B  = 


Pi  0  0 

0  Pt  0 
0  0  Pt 


(3) 


Empirically,  we  observe  that  for  elongated  ferrous  objects  such  as  cylinders  and  most  UXO,  the 
longitudinal  coefficient  is  greater  than  the  transverse  coefficient.  For  flat  ferrous  objects  such  as 
disks  and  plates,  the  opposite  is  true.  This  matches  the  behavior  of  these  objects  in  the 
magnetostatic  limit.  For  non-ferrous  objects  such  as  aluminum  cylinders  and  plates,  these 
relationships  are  reversed. 

We  tested  several  implementations  of  this  model  in  the  Requirements  Demonstration.  All  were 
designed  to  take  advantage  of  the  fact  that  we  obtain  reliable  position  (x,y,z)  information  from 
the  magnetometer  signals.  We  then  fitted  the  pulsed-induction  response  to  models  with 
combinations  of  two  or  three  response  coefficients,  P,  and  two  or  three  orientation  angles.  One 
of  the  goals  of  the  Requirements  Demonstration  was  to  determine  which  of  these  models  resulted 
in  the  most  classification  utility  with  the  least  data  collection  expense.  As  we  will  discuss  below, 
we  have  determined  that  collecting  two  orthogonal  EM  surveys  and  fitting  the  data  using  the 
three  p,  three  angle  model  yields  the  optimum  results.  This  is  the  survey  methodology  that  was 
used  in  this  Demonstration. 


2.2  Strengths,  Advantages,  and  Weaknesses  of  the  Technology 

No  single  method  currently  available  will  be  the  “magic  bullet”  of  classification.  We  have 
already  demonstrated^’’  that  an  impressive  level  of  discrimination  is  possible  using  the  standard 
MTADS  if  a  small  training  area  is  investigated  prior  to  data  analysis  on  the  entire  site  and  the 
distribution  of  ordnance  is  limited.  This  discrimination  is  based  primarily  on  fitted  dipole  “size.” 
In  this  program  we  have  demonstrated  methods  designed  to  add  an  extra  “dimension”  to  the 
discrimination,  that  of  “shape.”  For  items  of  the  same  induced  magnetic  dipole  we  can 
discriminate  based  on  the  ratio  of  responses  of  the  items  three  axes  to  the  EM  induction  sensors 
in  the  MTADS  suite.  As  we  will  show  in  a  later  section,  this  adds  some  discrimination  capability 
to  the  system. 

Even  with  the  most  optimistic  result  however,  these  methods  will  not  result  in  a  perfect  system. 
As  we  have  stated  above,  this  program  is  based  on  the  idea  of  classification  by  shape.  By 
definition,  this  implies  that  clutter  items  that  have  similar  shapes  to  ordnance  will  be  classified  as 
ordnance.  Items  such  as  pipes  and  post  sections  are  representative  of  this  problem.  If  it  is 
important  to  reduce  remediation  costs  to  the  extent  that  these  items  are  not  dug,  other  methods. 
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possibly  sensitive  to  composition  or  the  presence  of  explosive  compounds,  will  have  to  be 
employed  in  conjimction  with  those  being  developed  in  this  program. 

2.3  Factors  Influencing  Cost  and  Performance 

Implementation  of  the  methods  developed  in  this  program  requires  additional  survey  time 
compared  to  a  minimum  detection  survey.  We  have  previously  demonstrated  that,  in  many 
cases,  the  MTADS  can  detect  essentially  all  UXO  with  a  total-field  magnetometry  survey.  For 
ordnance  target  sets  that  include  60-  and  81 -mm  mortars  at  depths  of  0.75  to  1  m  an  overlapping 
EM  induction  survey  is  required  to  get  a  high  detection  probability.  With  the  current  MTADS 
EM  array  configuration,  we  require  two  orthogonal  EM  surveys  to  ensure  sufficient 
“illumination”  of  each  target  to  get  a  reliable  fit  to  the  model  presented  in  this  report.  This 
increases  the  survey  hours  on-site  considerably  although  it  does  not  impact  the  mobilization  and 
data  analysis  costs.  In  many  cases,  these  extra  costs  are  only  equivalent  to  the  cost  of  digging 
one  or  two  targets  per  acre. 


3.  Site/Facility  Description 


3.1  Background 

The  Requirements  Demonstration  was  performed  at  the  Naval  Research  Laboratory  Ordnance 
Classification  Test  Site  located  on  the  Army  Research  Laboratory’s  Blossom  Point  Research 
Facility.  This  site  was  designed'^  to  contain  a  variety  of  ordnance,  ordnance  simulants  and 
clutter  at  known  positions  and  orientations  for  algorithm  development  and  testing.  The  Final 
Demonstration  was  performed  on  a  live  range  at  the  Blossom  Point  Facility,  the  “L”  Range. 

During  World  War  H,  Harry  Diamond  and  his  team  at  the  National  Bureau  of  Standards  (NBS, 
now  named  NIST)  needed  open  areas  where  they  could  test  the  fuzes  they  were  developing. 

They  established  test  sites  at  Aberdeen  Proving  Ground,  MD,  Fort  Fisher,  NC  and  in  early  1943 
NBS  leased  land  and  established  a  proving  ground  for  proximity  fuzes  at  Blossom  Point.  By 
September  1945,  14,000  rocket  and  mortar  roxmds  had  been  fired.  In  1953,  the  lease  on  the 
property  was  transferred  to  the  Army,  which  operated  the  property  as  a  fast-reaction,  low-cost 
range  for  experimental  work.  Firing  ranges  provided  a  2000-yard  maximum  range  for  land 
impact  and  a  10,000-yard  maximum  for  water  impact.  During  the  Vietnam  War,  the  Army’s 
Harry  Diamond  Laboratory  was  very  active  at  the  site. 

The  “L”  Range  is  the  main  range  for  impact  testing  of  various  munitions  at  Blossom  Point.  It  is 
approximately  800  feet  wide  by  5000  feet  long  and  encompasses  ~93  acres.  This  range  has  been 
the  primary  impact  area  throu^out  the  history  of  the  site.  Some  of  the  known  firings  include 
81-mm  mortars  in  1961,  2.75-inch  rockets  fired  fi-om  helicopters  throughout  the  70’s,  and 
various  listings  of  60-nim  mortars,  75-mm  projectiles,  81-mm  mortars  and  various  barrage 
rockets. 

HFA,  Inc.  conducted  an  ordnance  removal  at  the  Blossom  Point  Test  Facilily  in  1996.*^  Two 
sites  were  cleared  in  conjunction  with  utility  work  and  construction.  Two  sites  totaling  66  acres 
were  cleared;  one  a  clear  area  parallel  to  “L”  range  and  one  a  wooded  area  north  of  the  first. 
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The  area  was  excavated  to  4  feet  on  the  construction  sites  and  2  feet  for  the  utility  lines.  Seven 
hundred  fifty  three  UXO  items  and  9,267  lbs.  of  scrap  were  removed  from  the  site.  The  UXO 


consisted  of  a  wide  variety  of  ordnance  types 
and  classes  with  a  preponderance  of  20-  and 
30-mm  rounds,  60-  and  81 -mm  mortars,  and 
4.2-in  rockets.  This  is  consistent  with  the 
firing  records  mentioned  above. 

3.2  Site  Characteristics 

Figure  2  is  a  road  map  of  a  portion  of  Charles 
Coimty,  Maryland  showing  the  location  of 
Blossom  Point  relative  to  La  Plata,  the  Coimty 
Seat.  Figure  3  is  an  aerial  photo  of  the 
Blossom  Point  Field  Facility  with  the  area 
comprising  the  Final  Demonstration  test  site 
highlighted. 

The  Demonstration  area  is  located  on  high 
groimd,  well  above  the  surrounding  rivers. 

The  site  has  good  sky  view  for  GPS  but  is 
bordered  by  a  densely  wooded  area  that  is  ideal 
for  testing  non-GPS  location  systems. 


Figure  2  -  Road  map  showing  the  location  of 
Blossom  Point. 
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Figure  3  -  Aerial  view  of  Blossom  Point,  MD  with  the  approximate 
location  of  the  Demonstration  Site  on  the  Army  Research  Laboratory 
Blossom  Point  "L"  Range  highlighted. 
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4.  Demonstration  Approach 


4.1  Performance  Objectives 

The  objective  of  the  Demonstration  was  to  quantify  the  classification  performance  available 
using  the  MTADS  array  of  modified,  off-the-shelf  pulsed-induction  sensors  and  the  data 
modeling  algorithms  developed  in  this  program.  The  Demonstration  proceeded  in  three  phases: 
data  collection' data  analysis,  and  target  marking  and  remediation. 

Data  collection  consisted  of  surveying  an  approximately  three-acre  area  on  a  live  range,  known 
to  have  had  many  detonations,  using  both  the  magnetometers  and  pulsed-induction  sensor  suites 
of  MTADS.  The  magnetometer  survey  was  conducted  in  an  E-W  orientation  to  minimize  the 
effects  of  GPS  signal  loss  at  the  woodline.  The  pulsed-induction  survey  was  carried  out  both  E- 
W  and  N-S  to  get  the  best  possible  “illumination”  of  each  target. 

Data  analysis  was  accomplished  using  the  MTADS  Data  Analysis  System.  The  DAS  has  been 
upgraded  during  this  program  to  simultaneously  analyze  a  magnetometer  and  several  pulsed- 
induction  survey  data  sets.  The  analysis  consists  of  fitting  individual  target  signatures  to  the 
model  discussed  above  to  extract  target  position,  size,  and  relative  response  along  three 
orthogonal  axes.  These  relative  response  coefficients  were  used  to  classify  the  target  as  UXO  or 
scrap.  This  phase  of  the  Demonstration  resulted  in  a  spreadsheet-like  target  report  that  included 
target  number,  location,  depth,  three  response  coefficients,  and  predicted  class.  This  spreadsheet, 
and  the  reasoning  behind  the  target  assignments,  was  communicated  to  the  Institute  for  Defense 
Analyses  (IDA)  before  remediation  was  begun. 

The  final  phase  of  the  Demonstration  consisted  of  remediation  of  the  analyzed  targets.  We  had 
planned  to  select  approximately  200  targets  for  remediation.  After  analysis  of  the  survey  data  we 
found  that  there  were  only  ~200  targets  in  the  survey  area  with  signatures  well  enough  separated 
to  get  a  good  model  fit  so  no  selection  was  necessary.  This  target  set  represents  -25%  of  those 
targets  with  magnetic  anomaly  >  50nT  and/or  EM  anomaly  >  70nT.  In  our  view,  this  is  a  large 
enough  fi-action  of  the  total  targets  to  ensure  that  a  representative  sample  of  all  targets  was 
remediated.  These  targets  were  flagged  and  remediated  by  a  commercial  UXO  firm. 

4.2  Physical  Setup  and  Operation 

Since  this  Demonstration  was  conducted  on  the  Blossom  Point  site  adjacent  to  where  our 
equipment  is  housed,  many  of  the  normal  pre-survey  logistics  such  as  establishment  of  first-order 
GPS  markers,  transport  of  the  equipment  to  the  site,  and  equipment  setup  and  testing  were  not 
required.  We  performed  the  Demonstration  “out  of  the  garage.”  In  all  other  ways,  this 
Demonstration  was  conducted  in  accordance  with  our  normal  survey  practices.  The  actual 
demonstration  schedule  was: 
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Date 

29  July  1999 
3  August  1999 


4  August  1999 

12-13  August  1999 
16-18  August  1999 
26  August  1999 


Activity 

Magnetometer  Survey  of  Fusion  Site  and  Man- 
portable  Extension. 

East-West  EM  Survey  of  Fusion  Site  and  MP 
Extension. 

North-South  EM  Survey  of  a  portion  of  the 
Fusion  Site 

North-South  EM  Survey  of  remaining  Fusion 
Site  and  MP  Extension. 

Data  Analysis  and  Target  Classification 
Flag  Targets  for  Remediation. 

Target  Remediation. 

Required  Demolition.  81  Shots  on  73  Targets. 


Required  Time 
2  hrs  Survey  Time 

5  hrs  Survey  Time 

1  hr  Survey  Time 

4  hrs  Survey  Time 


16  Man  Hours 
12  Man  Days 


5.  Performance  Assessment 


Although  not  part  of  the  Final  Demonstration  of  this  project,  data  will  be  presented  in  this 
section  that  was  collected  during  the  development  of  the  algorithms  and  in  the  Requirements 
Demonstration  in  February  1999.  These  data  will  illustrate  the  performance  of  the  model 
presented  here  under  idealized  conditions  and  will  allow  useful  conclusions  to  be  drawn  about 
the  performance  degradation  suffered  when  real-world  sensor  noise,  location  uncertainty,  and 
finite  target  density  are  encountered. 

5.1  Performance  Data 

5.1.1  Requirements  Demonstration 

The  NRL  Ordnance  Glassification  Test  Site  contains  a  series  of  ferrous  and  non-ferrous  shapes, 
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selected  clutter  items  and  inert  ordnance  buried  at  known  locations,  orientations,  and  depths. 
During  development  of  the  model,  EM  induction  signatures  were  acquired  for  many  of  the  items 
to  be  emplaced  in  the  test  field  on  a  laboratory  jig  that  held  the  items  in  precisely  known 
positions  and  orientations.  During  the  Requirements  Demonstration,  survey  data  were  collected 
on  the  test  field  using  a  variety  of  deployment  approaches  and  MTADS  EM  array  orientations  to 
determine  the  most  cost-effective  approach  for  classification.  For  the  purposes  of  this  report,  we 
will  only  consider  the  data  set  that  consists  of  combined  N-S  and  E-W  survey  lines.  The  results 
obtained  from  other  combinations  of  survey  conditions  are  similar. 

As  an  illustration  of  the  performance  of  the  model  developed  in  this  work  compared  to  the  sphere 
model  which  is  the  basis  of  the  initial  MTADS  DAS,  let  us  focus  on  one  of  the  targets  in  the  test 
field,  an  81 -mm  mortar  at  0.5  m  depth.  The  individual  survey  tracks  over  this  target  are  shown 
in  Figure  4.  At  each  of  the  locations  shown,  a  reading  is  acquired  from  the  sensors.  The 
measured  data  and  model  fits  fi*om  the  3P  model  and  the  sphere  model  are  shown  in  Figure  5  for 
each  of  the  two  sxirveys.  As  can  be  seen  in  the  left-hand  panel  of  the  Figure,  the  signature  fi'om  a 
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Figure  4  -  MTADS  survey  tracks  over  an  81 -mm  mortar  placed  parallel  to  the  surface  at  a 
depth  of  0.5  m.  The  right  sensor  is  represented  by  green  symbols,  the  center  by  black  and  the 
left  by  red. 


North-South  Survey  East-West  Survey 


y  (m)  y  (m) 

Figure  5  -  Measured  response  and  model  fits  using  both  the  sphere  model  and  the  3p  model 
developed  for  this  program  for  an  81 -mm  mortar  at  0.5  m.  The  color  scheme  for  the  three 
sensors  in  the  MTADS  array  is  as  in  Figure  4. 
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target  oriented  along  track  is  the  now  familiar  double-humped  shape.  This  arises  from  the 
varying  coupling  of  the  transmitted  pulse,  as  shown  in  Figure  1,  with  the  target.  As  the  array 
approaches  a  target  oriented  along  track,  the  coupling  with  the  long  axis  of  the  target  is  strong, 
although  the  distance  is  large,  and  the  signal  is  large.  Directly  over  the  target,  the  coupling  is  to 
the  short  axis  of  the  target  and  the  signal  is  relatively  weak.  Of  course,  the  sphere  model  can  not 
reproduce  this  asymmetry.  The  right-hand  panel  depicts  the  target  oriented  across  track.  Here 
the  sphere  mo^  can  reproduce  the  observed  shape  but  the  fitted  depths  are  not  correct.  The 
results  are  similar  for  the  mortar  placed  vertically;  the  shape  is  symmetric  but  the  sphere  model 
does  not  return  the  correct  depth. 

An  overview  of  the  entire  Test  Field  is  shown  in  Figure  6.  The  measured  anomalies  associated 
with  each  of  the  61  targets  in  the  test  field  were  isolated  and  fit  to  the  model  as  above.  Sample 


Figure  6  -  North-South  EM  survey  of  the  Test  Field. 


11 


fit  results  for  various  test  shapes  are  presented  in  Table  I.  Two  goals  of  this  test  were  to 
determine  how  the  beta  coefficients  varied  with  object  shape  and  how  much  measurement  errors 
would  effect  the  absolute  and  relative  values  of  the  fitted  betas  compared  to  those  obtained  in  a 
static  jig  test  of  the  items  emplaced  in  the  test  field. 


Table  I.  Model  Fits  and  Trial  Classification  of  Target  Shape  for  Sample  Test  Field  Targets 


Item 

P2 

P3 

Classification. 

3"d  X  6"1  steel  cylinder 

2.063 

1.195 

1.100 

LONG 

3"d  X  12"1  steel  cylinder 

5.74 

2.76 

2.526 

LONG 

3"d  X  24"1  steel  cylinder 

8.634 

4.475 

3.828 

LONG 

8"  X  8"  X  14"  square  steel  plate 

4.954 

4.577 

1.499 

FLAT 

4"  X  4"  X  54"  square  steel  plate 

1.142 

0.906 

0.303 

FLAT 

12"  X  3"  X  54"  rectangular  steel  plate 

3.427 

0.796 

0.422 

LONG 

4.8"d  steel  sphere 

3.093 

2.217 

2.029 

LONG 

81 -mm  Mortar 

5.373 

0.842 

0.721 

LONG 

Shovel  Blade 

6.315 

5.806 

3.386 

FLAT 

Depending  on  these  two  factors,  it  was  initially  thought  that  significant  discrimination  could  be 
achieved  by  simply  relating  the  relative  betas  directly  to  object  shape,  i.e.  Pi  >  P2  »  Ps  implies  a 
long,  ordnance  shape  and  Pi  »  P2  >  P3  implies  a  flat,  debris  shape.  Overall,  we  correctly 
classified  33  of  the  34  targets  that  were  ordnance  or  ordnance  simulants  as  being  long  and  10  of 
the  25  plate  and  clutter  items  as  being  flat.  Thus,  using  this  model  we  are  able  to  avoid 
identifying  40%  of  the  debris  as  ordnance  (false  positives)  while  only  missing  one  ordnance  item 
(false  negative)  in  this  idealized  test  field. 

One  problem  identified  with  this  simple  shape  based  discrimination  arises  with  rectangular 
plates.  Despite  the  distinct  relative  dimensions  of  the  12"  x  3"  x  V*'  plate  in  Table  I,  the  range  of 
measured  betas  does  not  directly  reflect  this:  3.427:0.796:0.422.  The  relative  betas  are  more 
indicative  of  a  long,  ordnance  type  item  than  flat,  clutter  item. 

The  variability  of  the  fitted  betas  for  one  of  the  test  targets,  a  3"d  by  12"1  steel  cylinder,  at 
different  depths  and  orientations  is  shown  in  Table  II.  For  each  beta,  there  is  a  20-30%  variation 
about  the  mean.  Related  to  this  is  the  fit  result  for  a  steel  sphere  in  Table  I.  While  it  is  expected 
that  the  fitted  betas  should  be  equal,  they  are  not.  The  range  in  the  three  values  is  again  roughly 
25%  about  the  mean.  This  variation  presumably  arises  fi-om  measurement  error  (sensor  noise 
and  positioning  error)  and  it  places  a  limit  on  how  well  the  relative  beta  values  can  be  used  to 
discriminate  shape  based  on  relative  values  and  individual  objects  based  on  absolute  values. 
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Table  II.  Model  Fits  for  the  four  3"  D  x  12"  L  Steel  Cylinder  at  the  Test  Field 


3i 

Pa 

Depth  (m) 

Orientation 

5.740 

2.760 

2.526 

0.35 

Horizontal 

3.918 

2.733 

2.051 

0.35 

Vertical 

2.052 

1.804 

0.5 

Horizontal 

6.297 

1.777 

1.058 

0.5 

Vertical 

This  variability  in  relative  betas  is  illustrated  in  Figure  7.  The  left-hand  panel  of  the  figure  is  a 
ternary  plot  of  the  betas  derived  fi'om  measurements  on  the  test  field  objects  on  a  test  jig  fit 
assuming  axial  symmetry.  We  did  not  collect  enough  data  at  this  time  to  allow  a  fit  without 
assumption  of  symmetry.  As  expected,  the  long  symmetric  objects  (cylinders  and  ordnance) 
exhibit  one  large  beta  and  two  smaller  betas  and  the  plates  have  one  small  and  two  large 
responses.  The  right-hand  panel  shows  the  same  plot  for  betas  derived  fi-om  the  fits  to  the  survey 
data.  The  effect  of  sensor  noise  and  location  uncertainty  is  to  broaden  the  long  objects  off  the 
diagonal  and  partially  confuse  them  with  the  rectangular  plates. 


Pi  Pi 

Figure  7  —  Ternary  plots  of  the  relative  values  of  the  three  betas  derived  from  fits  of  measured 
signatures.  The  left-hand  panel  is  the  result  of  measurements  in  a  test  jig.  The  right-hand  panel  results 
from  field  measurements  and  shows  the  effects  of  measurement  errors  as  discussed  in  the  text. 

One  measure  of  performance  in  detection  and  discrimination  problems  is  a  Receiver  Operating 
Characteristic  (ROC)  curve.  These  curves  plot  the  probability  of  detection  (Pd)  versus  the 
probability  of  false  alarm  (Pfa)  as  some  threshold  of  detection  is  varied.  In  the  case  of 
ordnance/clutter  discrimination,  the  curve  plots  the  probability  of  correctly  identifying  an 
ordnance  item  as  ordnance  versus  the  probability  of  incorrectly  identifying  a  clutter  item  as 
ordnance.  The  threshold  level  that  is  varied  is  the  range  of  the  discriminating  fit  parameter.  A 
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ROC  curve  for  this  analysis  methodology  can  be  generated  by  creating  an  area  wdth  Pi  >  40  and 
P2  »  P3  and  then  progressively  expanding  the  width  of  this  area.  This  ROC  curve  is  shown  in 
Figure  8.  Approximately  one  third  of  the  false  alarms  can  be  eliminated  wdthout  reducing  Pd- 


Figure  8  -  ROC  curve  generated  using  3P  analysis  on  the 
survey  data  from  the  Ordnance  Classification  Test  Field. 

For  all  61  targets  in  the  test  field  the  model-derived  locations  were  a  fraction  of  the  data  spacing 
(0.15m  along  track  and  0.5m  sensor  separation).  Depth  estimates  were  a  fraction  of  the  object 
size.  For  most  targets  the  derived  orientations  agreed  well  with  the  known  values.  The 
exceptions  were  symmetric  objects  oriented  vertically.  These  targets  should  have  a  symmetric 
anomaly.  At  the  time  these  data  were  collected,  we  had  a  small  timing  error  among  the  three 
sensors  in  the  array.  The  result  of  this  timing  error  was  to  introduce  some  artifact  asymmetry 
into  the  anomalies.  Ms  timing  error  has  since  been  eliminated. 


5.1.2  Final  Demonstration 

After  completion  of  a  magnetometer  and  two  EM  surveys  in  perpendicular  directions,  201  targets 
were  analyzed  and  marked  for  remediation  on  the  L  Range  Final  Demonstration  site.  An 
abbreviated  version  of  the  MTADS  target  report  for  these  items  is  attached  as  Appendix  C.  A 
more  complete  version  of  the  report  with  positions  in  absolute  coordinates  and  the  results  of  the 
trial  2P  analysis  has  been  provided  to  EDA  and  is  available  from  the  authors. 

A  total  of  1 88  targets  were  recovered  from  this  test  area.  Examples  are  pictured  in  Figure  9. 
There  were  66  ordnance  items,  20  ordnance  related  items,  66  exploded  fragments,  and  36  items 
not  related  to  ordnance.  The  ordnance  items  broke  down  into  three  groups:  48  81 -mm  mortars,  8 
mortars  of  smaller  sizes,  and  10  miscellaneous  ordnance  items.  The  miscellaneous  items 
included  2  bomb  fuzes,  a  76mm  projectile,  and  two  5-in  rockets.  The  ordnance  related  items 
were  rocket  motors  with  fins  and  mortar  tail  booms.  The  exploded  fragments  appeared  to  be 
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non-ordnance 

Figure  9  -  Examples  of  the  items  remediated  during  the  Final  Demonstration  of  the  program. 
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mostly  from  mortars.  The  non-ordnance  items  included  cable  tie  down  points  for  test  towers  that 
had  been  removed,  block  and  tackles  from  the  cables,  and  a  variety  of  odd  scraps  of  metal  (rebar, 
sheet  metal,  angle  iron,  and  bolts).  We  will  concentrate  our  initial  discussion  on  the  81 -mm 
mortars  as  they  provide  the  best  fit  statistics.  After  remediation,  three  of  what  proved  to  be  81- 
mm’s  (targets  166,  171,  and  178)  were  found  to  have  3p  fits  in  which  the  primary  p  was  very 
large  and  one  of  the  secondary  p  ’s  was  near  zero.  Re-inspection  of  the  survey  data  showed  that 
this  resulted  froiii  a  large  zero  offset  between  the  two  perpendicular  survey  tracks.  After 
correcting  for  this  offset,  reasonable  p  fits  were  obtained  for  these  objects.  The  parameters  of 
these  corrected  fits  are  listed  in  Appendix  C  and  used  in  the  analysis  that  follows. 

The  results  of  the  three-beta  fits  for  the  81-mm’s  are  shown  in  Figure  10.  The  values  of  the 
primary  beta  (largest)  are  plotted  on  the  x-axis.  The  two  smaller  betas  are  plotted  on  the  y-axis, 
where  the  symbol  in  the  plot  represents  the  average  of  the  two  and  the  vertical  line  represents  the 
spread  between  the  two  values.  We  want  to  preserve  the  magnitude  of  beta,  which  was  lost  in 
the  ternary  plots  used  above.  We  find  this  to  be  an  easier  way  to  visualize  the  spread  in  the  data 
than  plotting  the  points  in  three  dimensions.  Note  that  if  the  fit  results  were  perfect  (no 
measurement  errors)  then  the  data  would  all  be  symbols  with  no  vertical  line  (secondary  betas 
are  equal  for  axisymmetric  objects). 


Figure  10  -  Two-dimensional  representation  of  the  three  beta  fits  for  the  81- 
mm  mortars  in  the  Final  Demonstration.  The  value  of  the  primary  beta 
(largest)  is  plotted  on  the  x-axis.  The  two  smaller  betas  are  plotted  on  the  y- 
axis,  where  the  symbol  represents  the  average  and  the  vertical  line  represents 
the  spread. 


The  three  beta  values  are  best  described  by  a  log-normal  distribution.  In  logarithmic  quantities, 
the  mean  is  0.6970,  0.3183,  and  0.3098  with  standard  deviations  of  0.2,  0.09,  and  0.13  for  pi,  p2, 
and  p3  respectively.  In  measured  units,  this  corresponds  to  an  average  response  of  4.98  along  the 
length  of  the  mortar  and  2.0  transverse  to  this.  Note  that  the  value  range  from  2  to  12  along  the 
primary  axis,  which  is  much  greater  than  the  20-30%  observed  for  objects  in  the  test  field.  It  is 
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thought  that  this  enlarged  spread  is  largely  due  to  positioning  errors  in  height  as  the  array  is 
towed  over  the  uneven  ground  of  a  live  site.  We  will  discuss  this  point  in  more  detail  later  in  the 
report.  The  ellipse  plotted  in  Figure  10  represents  a  three  dimensional  ellipsoid  with  major  and 
minor  radii  that  are  equal  to  two  standard  deviations  of  the  primary  and  secondary  betas.  The 
ellipse  is  tilted  because  of  a  weak  correlation  between  the  primary  versus  the  secondary  betas 
(stronger  primary  betas  have  stronger  secondary  betas).  As  we  will  show  below,  this  ellipsoidal 
curve  can  be  used  to  calculate  the  probability  that  a  given  beta  fit  represents  an  81 -mm  mortar. 

The  three  beta  fit  results  fi-om  the  other  ordnance  items  recovered  at  the  L  Range  are  plotted  in 
the  left  panel  of  Figure  11.  The  approximate  primary  versus  secondary  beta  values  range  fi-om 
0.7/0.5  for  the  bomb  fuze  to  178.0/62.0  for  the  5-in  rockets.  A  similar  plot  for  the  fiagments,  the 
ordnance  related  scrap,  and  the  non-ordnance  scrap  is  presented  in  the  right  panel  of  Figure  11. 

It  is  interesting  to  note  that  the  bulk  of  the  fiagments  do  not  overlap  the  81 -mm  mortars.  One 
would  expect  that  a  large  spread  in  the  secondary  betas  should  result  fiom  an  irregularly  shaped 
object.  Overall,  the  spread  observed  in  the  right  panel  of  Figure  1 1  is  not  much  worse  than  the 
spread  for  the  axisymmetiic  ordnance  objects  (Figure  10  and  left  panel  of  Figure  1 1).  After 
examining  photos  of  the  objects  dug,  this  is  not  too  surprising.  Most  of  the  scrap  is  to  first  order 
elongated,  with  approximately  equal  secondary  dimensions. 
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Figure  1 1  -  Two-dimensional  representation  of  the  three  beta  fits  for  targets  from  the  Final 
Demonstration  plotted  as  in  Figure  10.  The  left  panel  shows  the  results  for  all  other  ordnance 
recovered  from  the  site,  and  the  right  panel  those  for  the  scrap  and  clutter  recovered. 

Examples  of  ROC  curves  based  on  the  L-Range  data  are  shown  in  Figures  12  and  13.  To 
generate  these  curves,  the  ellipsoid  in  Figure  10  is  grown  larger  (in  three  dimensions)  and  the 
number  of  ordnance  (PD)  and  non-ordnance  (FA)  betas  that  fall  within  this  three  dimensional 
region  are  counted.  Figure  12  plots  the  results  for  a  single  81 -mm  ellipsoid.  In  Figure  13, 
ellipsoids  are  generated  about  each  of  the  ordnance  items  present.  The  sizes  of  these  ellipsoids 


are  grown  uniformly  based  on  the  standard  deviations  and  correlations  of  the  81 -mm  betas;  too 
few  of  these  other  ordnance  items  were  fitted  to  generate  valid  beta  statistics.  This  is  illustrated 
on  the  familiar  beta  plot  in  Figure  14. 


Figure  12  -  ROC  curve  for  the  detection  of  81 -mm  mortars 
from  the  Final  Demonstration 


Figure  13  -  ROC  curve  for  detection  of  all  ordnance  at  die 
Final  Demonstration 
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Figure  14  -  Two-dimensional  representation  of  the  three  beta  fits  to  all  targets 
from  the  Final  Demonstration  with  ellipses  for  each  ordnance  class  derived 
from  the  8 1  -mm  mortar  ellipse 

The  discrimination  performance  for  a  single  ordnance  item,  81 -mm  mortars,  matches  that 
observed  in  the  test  field.  We  achieve  a  roughly  40%  reduction  in  false  alarms  without 
impacting  Pp.  The  story  is  more  complicated  when  trying  to  discriminate  several  classes  of 
ordnance  fi’om  the  background  clutter.  Figure  13.  In  order  to  identify  the  small  fuzes  in  this  field 
as  ordnance,  a  large  number  of  clutter  items  have  to  be  included.  In  part,  this  is  the  inevitable 
result  of  trying  to  discriminate  ordnance  ranging  in  size  from  fuzes  to  5-in  rockets  from  clutter. 
This  difficulty  may  be  mitigated  by  obtaining  more  data,  hence  better  fit  statistics,  on  the  smaller 
ordnance  items.  Using  the  error  ellipsoid  derived  from  the  distribution  of  81 -mm  mortar  fits,  as 
we  were  forced  to  do,  may  well  overstate  the  region  of  the  3-D  space  occupied  by  the  smaller 
ordnance  items.  As  we  obtain  more  model  fits  to  remediated  ordnance  and  improve  our  fit 
statistics  we  will  be  able  to  test  this  notion. 

5.2  Data  Assessment 

The  survey  data  collected  at  the  Final  Demonstration  of  this  program  were  of  sufficient  quality  to 
meet  the  goals  of  the  Demonstration.  We  were  able  to  increase  the  discrimination  available 
using  MTADS  EM  induction  survey  data.  Several  features  of  the  data  limited  the  classification 
ability  however.  We  showed  in  the  Requirements  Demonstration  that  sensor  noise  and  sensor 
location  error  limited  the  estimated  betas  to  a  precision  of  ~25%.  Some  improvement  is  possible 
in  this  regard  but  not  a  lot.  The  GPS  units  used  for  sensor  location  on  the  MTADS  array  are 
state-of-the-art  receivers  with  cm-level  precision.  Because  of  the  response  of  the  EM-61  sensors 
to  the  GPS  antenna,  the  antenna  is  located  ~1 .5  m  in  front  of  the  sensor  array.  Although  the 
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antenna  location  is  known  to  centimeters,  there  is  some  location  imcertainty  introduced  by  the 
back  projection  of  the  sensor  locations  from  the  antenna  position.  A  two-antenna  array,  with  a 
GPS  antenna  in  front  of  and  behind  the  EM  sensors,  would  reduce  sensor  location  uncertainties. 
At  the  time  of  this  Demonstration,  this  would  have  involved  the  purchase  of  another  independent 
GPS  receiver/radio  combination.  Now,  because  of  the  demand  from  the  mining  and  construction 
markets,  dual  antenna  systems  are  available  for  a  modest  increase  in  price.  Sensor  noise  is  a 
different  issue.^rogress  here  requires  a  new  generation  of  EM  induction  sensors. 

Compared  to  the  data  collected  during  the  Requirements  Demonstration,  there  was  a  further 
decrease  in  the  precision  of  the  fitted  beta  values  during  the  Final  Demonstration.  We  attributed 
this  above  to  vertical  motion  of  the  EM  array  over  the  rough  ground  at  the  live  site.  In  an 
attempt  to  provide  some  quantitative  underpinning  to  this  assertion,  we  have  performed  a  Monte 
Carlo  simulation  of  the  fitted  response  of  an  81 -mm  mortar  simulant  with  varying  sources  and 
magnitudes  of  noise.  The  object  used  in  the  simulation  had  a  primary  beta  of  5  and  two 
secondary  betas  of  2,  about  that  expected  for  an  81 -mm  mortar.  The  object  was  placed  at  a 
distance  of  0.6  m  from  the  sensor  array  and  given  a  random  x,y  position  relative  to  the  survey 
tracks  and  a  random  orientation.  Each  simulation  included  real  MTADS  GPS  and  sensor  noise. 
The  results  of  this  simulation  are  shown  in  Figure  15.  The  top  panel  shows  the  results  using  only 
GPS  and  sensor  noise.  In  this  case,  the  fitted  betas  exhibit  just  the  precision  observed  in  the 
Requirements  Demonstration,  -25%.  For  the  simulation  depicted  in  the  bottom  panel  of  Figure 
15,  a  component  of  sensor  height  variation  was  added  to  simulate  array  bouncing  over  rough 
ground.  We  find  that  red  noise  with  an  RMS  amplitude  of  3  cm  reproduces  the  spread  in  betas 
observed  in  the  Final  Demonstration.  This  is  easily  within  the  realm  of  possibility;  the  MTADS 
EM  array  platform  does  not  have  a  suspension  and  is  observed  to  bounce  in  rough  terrain. 

The  terrain  at  the  L  Range  demonstration  was  not  especially  rough  for  a  live-site  demonstration; 
MTADS  has  been  demonstrated  at  several  sites  with  much  more  challenging  terrain.  Therefore, 
to  take  advantage  of  the  shape  information  inherent  in  the  response  of  targets  to  the  EM-61  array, 
better  control  of  vertical  displacements  will  be  required.  One  option  is  to  add  suspension  to  the 
array  platform.  Another,  possibly  more  effective,  method  would  be  to  record  the  displacements 
of  the  array  using  inertial  sensors  and  explicitly  accoxmt  for  the  position  of  the  array  in  three 
dimensions  in  the  data  analysis  procedure. 

5.3  Technology  Comparison 

The  obvious  baseline  for  comparison  of  the  value  of  the  technology  demonstrated  here  is  the 
current  MTADS.  As  mentioned  above,  the  baseline  MTADS  is  able  to  achieve  a  reasonable  level 
of  discrimination  using  magnetometry  fits  alone,  especially  when  the  ordnance  distribution  is 
limited.  We  will  thus  compare  the  results  obtained  in  this  demonstration  with  those  that  would 
be  obtained  by  MTADS  at  the  same  site.  To  accomplish  this,  we  have  made  use  of  the  fitted  mag 
“size”  parameter  that  is  included  in  the  target  report  in  Appendix  C.  For  each  ordnance  class,  we 
calculate  a  mean  “size.”  Just  as  in  the  case  of  the  3-beta  algorithm,  we  are  able  to  calculate  a 
distribution  about  this  mean  for  the  81 -mm  mortars.  We  use  the  81 -mm  distribution  to  generate 
a  proportionally  sized  distribution  for  each  ordnance  class.  The  distributions  derived  are  listed  in 
Table  Iff. 
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Figure  15  -  Results  of  a  Monte  Carlo  simulation  of  the  fitted  betas  resulting  from  a  range  of 
model  SI -mm  mortars.  The  top  panel  includes  MTADS  GPS  and  sensor  noise  only.  The 
middle  panel  adds  vertical  noise  to  simulate  bouncing  of  the  array  over  rough  terrain.  The 
lower  panel  shows  results  from  the  Demonstration  for  reference. 


Table  HI  -  Size  Distributions  used  in  Magnetometer  Analysis  of  the  Final  Demonstration 


Ordnance  Class 

Size  Distribution  (mm) 

Ordnance  Class 

Size  Distribution  (mm) 

Fuzes 

43  ±9 

81 -mm  mortar 

76  ±16 

Mk23 

56  ±  11 

105 -mm  projectile 

105  ±21 

60-mm  mortar 

60  ±  12 

5" rocket 

212  ±42 

We  can  then  generate  a  ROC  curve  for  this  method  by  varying  the  width  of  the  distribution 
around  each  ordnance  class  and  declaring  each  target  as  ordnance  (within  the  six  size  bands)  or 
clutter.  The  result  of  this  analysis  is  plotted  in  Figure  16.  Also  plotted  in  the  Figure  is  a  curve 
generated  by  enhancing  the  magnetometry  analysis  by  taking  advantage  of  the  fitted  magnetic 
dipole  orientation  for  each  target.  This  enhancement  relies  on  the  observation  that  UXO  targets 
have,  in  general,  been  shock  demagnetized  by  their  impact  with  the  ground  and  only  exhibit 
induced  magnetic  moments  while  fi'agments  and  clutter  have  remanent  moments.  This  was  the 
case  for  the  ordnance  recovered  at  the  L  Range,  only  one  of  the  73  items  considered  had  a 
magnetic  dipole  orientation  not  consistent  with  an  induced  dipole  only.  Note  that  this  method 
does  not  automatically  eliminate  all  items  with  a  remanent  moment,  only  those  whose  net  dipole 
orientation  is  outside  that  expected  fi-om  an  item  with  an  wholly  induced  dipole. 


Figure  16  -  ROC  curve  for  classification  at  the  L  Range  comparing 
the  results  obtained  with  the  3-beta  algorithm  to  those  using 
magnetic  dipole  "size"  and  dipole  orientation  enhancements 

The  magnetic  dipole  “size”  suffers  fix)m  many  of  the  same  problems  as  the  3-beta  algorithm 
when  attempting  to  discriminate  all  ordnance.  In  order  to  capture  the  fiizes,  a  large  number  of 
small  fi-ag  items  must  be  included.  The  magnetic  dipole  orientation  filter  helps  greatly  in  this 
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regard  as  a  good  number  of  the  frag  items  are  magnetized  and  are  thus  correctly  identified  as 
clutter. 

It  is  difficult  to  compare  the  performance  of  the  analysis  of  EM-61  data  presented  here  with  that 
of  other  sensors  and  analysis  methods.  As  we  have  shown,  the  current  procedure  gives  excellent 
results  in  the  test  jig  and  reasonable  results  at  our  Test  Field,  which  is  a  smooth,  clean,  and  level 
site.  The  onlydegitimate  comparison  is  to  results  obtained  by  competing  technologies  on  live- 
site  surveys.  As  these  data  become  available,  direct  comparisons  will  follow. 


6.  Cost  Assessment 


6.1  Cost  Performance 

The  estimated  costs  for  an  MTADS  EM  survey  in  two  directions  and  the  data  analysis  required  to 
implement  the  model  described  here  for  a  hypothetical  200  acre  survey  are  listed  in  Table  IV. 

Table  IV  -  Estimated  Costs  for  a  Hypothetical  200-Acre  Survey  Using  These  Methods. 


Mobilization  and  Logistics 


Activity 

$K 

10 

Equipment  Transport 

5 

Storage  and  Offices 

5 

Power  and  Fuel 

3 

Miscellaneous 

2 

Total 

.  25 

Survey  and  Analysis 
Activity  $K 

Surface  Clearance 


Field  Surveys 


Data  Analysis 


Target  Flagging 


Total 


is  Demobilization 

$K  Activity 

I 

30  Site  Cleanup 


200  Demobilization 


i 


Note  that  the  survey  costs  include  two  EM  surveys  only,  no  magnetometer  survey  is  included.  If 
large,  deep  targets  were  expected  a  magnetometer  survey  would  be  required  and  an  additional 
$80K  would  be  necessary.  Since  two  perpendicular  EM  surveys  are  included  in  the  estimate 
while  only  one  would  be  required  for  target  detection,  it  is  clear  that  the  added  cost  of  these 
methods  is  $500  per  acre.  This  is  approximately  equal  to  the  costs  required  to  remediate  three 
targets  per  acre.  Thus,  the  economic  breakeven  point  for  the  use  of  these  methods  is  reached 
when  tlu-ee  false  alarms  per  acre  are  avoided. 
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7.  Technology  Implementation 


7.1  DoDNeed 

Past  DoD  activities  have  contaminated  large  tracts  of  land  and  water  with  UXO.  It  is  estimated 
that  there  are  millions  of  acres  of  suspect  land  and  there  are  600  FUDS  and  44  BRAC  sites 
requiring  remediation.  This  UXO  contamination  prevents  civilian  use  of  the  land  and  will  thus 
require  remediation.  Approximately  $125  M/yr  is  being  spent  in  the  FUDS  and  BRAC  process. 
Thus,  this  is  a  large  problem  for  the  DoD  and  one  where  R&D  efforts  can  be  repaid  many  times 
over. 

The  cost  of  a  current  technology  UXO  remediation  operation  is  driven  by  false  alarms.  The 
Army  Corps  of  Engineers  has  estimated  that  during  the  clean  up  of  a  heavily  contaminated  5000- 
acre  site,  76%  or  $22.8M  of  the  required  funds  goes  to  non-UXO  removal.  Obviously,  any 
techniques  that  can  reduce  this  false  alarm  problem  should  be  investigated. 

7.2  Transition 

The  baseline  A/E4£)S' technology  has  been  transitioned  via  a  Cooperative  Research  and 
Development  Agreement  between  NRL  and  Blackhawk  Geometries  of  Golden  CO.  Blackhawk 
has  replicated  both  the  magnetometer  and  EM  arrays  using  the  engineering  drawings  and  plans 
provided  to  them  under  a  license  agreement  from  NRL.  The  baseline  MTADS  data  analysis 
system  was  also  transitioned  using  this  agreement.  Both  Blackhawk  and  NRL  have  continued  to 
improve  the  data  analysis  system  both  jointly  and  independently.  For  example,  both  groups  are 
now  running  the  DAS  on  a  LINUX-based  PC  instead  of  the  older  UNIX  work  stations.  Both 
groups  have  also  worked  over  the  past  two  years  to  extract  classification  information  from  EM 
induction  survey  data.  This  program  is  an  example  of  these  efforts. 

The  results  of  this  program  have  been,  and  will  continue  to  be,  communicated  to  the  UXO 
community  through  presentations  and  publications.  There  have  been  several  presentations  each 
of  the  past  two  years  at  the  UXO  Forum  and  we  plan  to  present  a  wrap-up  talk  next  year.  We 
have  submitted  a  paper  on  these  methods  and  the  results  of  the  Demonstrations  to  the  upcoming 
special  issue  of  the  IEEE  Transactions  on  Geoscience  and  Remote  Sensing  on  UXO.  By  these 
means,  we  mean  to  keep  the  entire  community  of  time-domain  EM  induction  sensor  users 
informed  of  our  progress,  not  just  the  users  of  the  commercial  MTADS. 

The  fitting  methods  from  this  program  will  be  used  in  part  during  the  Jefferson  Proving  Ground 
Demonstration  in  late  Summer  2000.  This  should  provide  a  good  test  of  these  methods  at  a  blind 
ordnance  detection/classification  exercise.  If  they  prove  successful,  we  are  confident  “they  will 
come.” 
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Appendix  A.  Points  of  Contact 


ESTCP 

Dr.  Jeffrey  Marqusee 

Ms.  Catherine  Vogel 

Institute  for  Defense  Analyses 
Dr.  Anne  Andrews 

NHL 

Dr.  Herbert  Nelson 

Dr.  J.  R.  McDonald 


ESTCP  Director 


Program  Manner,  Cleanup 


Analyst 


Principal  Investigator 


Deputy  Principal  Investigator 


Tel:  703-696-2120 
Fax:  703-696-2114 
marqusj  @acq.osd.niil 

Tel:  703-696-2118 
Fax:  703-696-2114 
vogelc@acq.osd.mil 


Tel:  703-578-2966 
Fax:  703-578-2877 
AAndrews@ida.org 


Tel:  202-767-3686 
Fax:  202-404-8119 
herb.nelson@nrl.navy.mil 

Tel:  202-767-3340 
Fax:  202-404-8119 
j  .mcdonald@nrl.navy.mil 


Army  Research  Laboratory,  Blossom  Point 

Mr.  Jack  Kaiser  Site  Manager  Tel:  301-870-2329 

Fax:  301-870-3130 
jkaiser@arl.mil 

Mr.  Bill  Davis  Explosives  Safety  Officer  Tel:  301-394-2434 

Fax:  301-394-2514 
wdavis@arl.mil 

Hughes  Associates,  Inc. 

Mr.  Richard  Robertson  Program  Manager  Tel:  202-767-3556 

Fax:  202-404-8119 
roberts5@ccf.nrl.navy.mil 


GeoCenters,  Inc. 


Mr.  Larry  Koppe 

AETC,  Inc.  si 
Dr.  Tom  Bell 

Dr.  Bruce  Barrow 


Site  Safety  Officer 


Tel:  301-753-1690 
Fax:  301-870-3130 
LarryEOD@aol.com 


Project  Manager 


Project  Scientist 


Tel:  703-413-0500 
Fax:  703-413-0505 
tbell@va.aetc.com 

Tel:  703-413-0500 
Fax:  703-413-0505 
bjb@va.aetc.com 


28 


Appendix  B.  Data  Archiving  and  Demonstration  Plan 

MTADS  survey  data  are  collected  as  a  collection  of  data  files  each  containing  the  time-stamped 
(DAQ  computer  time)  input  stream  from  the  individual  sensors.  These  input  streams  include 
GPS  UTC  time,  GPS  pulse-per-second  (PPS),  GPS  position,  sensor  trigger,  and  sensor  input 
data.  After  transfer  of  the  files  to  the  MTADS  DAS,  the  data  undergo  a  preprocessing  step  in 
which  all  times  ill  corrected  to  UTC  and  the  individual  sensor  positions  are  interpolated  from 
the  measured  positions  vs.  time  in  the  GPS  position  file.  Magnetometer  data  are  then  corrected 
by  removing  the  diurnal  variation  of  the  Earth’s  field  as  measured  by  a  reference  sensor  located 
near  the  survey  site  and  the  heading  error  resulting  from  residual  vehicle  signature.  EM  data 
have  the  long  term  baseline  drift  of  the  sensors  removed  by  subtracting  a  1000-point  running 
median  from  the  data.  At  this  point,  the  survey  data  are  available  as  background-corrected,  geo- 
referenced,  mapped  data  files. 

Both  the  raw  input  data  files  and  an  ASCII  version  of  the  mapped  data  file  are  archived.  The 
preprocessed  data  file  is  most  useful  for  other  investigators  and  these  files  are  maintained  on  a 
CD-ROM  by  the  principal  investigator.  These  data  files  and  a  copy  of  the  approved  test  plan  are 
available  by  contacting  Herb  Nelson  at  herb.nelson@nrl.navy.mil. 
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Appendix  C.  MTADS  Target  Report  from  the  Final  Demonstration 
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